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ABSTRACT 


A  eurrentt? problem  facing  the  cost  estimator  and  the  program 
manager  is  the  need  to  assess  program  cost  impacts  resultingfrom 
changes  in  production  rate.  ~fn  responding  to  this  problem ^Tihe 
Air  Force  Systems  Command  (AFSC)  has  developed  the  Production 
Rate  Model  for  providing  quick  same  day  responses. 

The  model  is  based  on  the  assumption  that  the  influence  an 
increasing  production  rate  has  on  decreasing  the  unit  cost  is 
limited.  This  idea  follows  the  minimum  cost  point  found  on  the 
long  run  average  cost  curve  in  economic  theory.  In  the  model's 
case  the  point  is  determined  by  the  manufacturer  prior  to 
production.  This  point  is  based  on  plant  capacity,  capital 
investment  and  manpower  requirements,  anticipated  quantities  and 
production  rate,  as^well-  as  requirements  specified  by  the 
government . 

The  AFSC  model  modifies  the  learning  curve  equation  by  including 
a  variable  representing  production  rate.  This  equation  is  used 
to  estimate  recurring  costs.  -In  addition i> There  is  an  internal 
data  base  of  historical  programs  containing  first  unit  cost, 
learning  curve  slope,  and  production  rate  slope  generated  with  a 
nonlinear  regression  technique.  The  nonlinear  regression 
technique  was  used  to  reduce  the  effects  of  multicollinearity 
and  bias  associated  with  ordinary  least  squares  regression., 
Virtually  all  tiuT  literature  addressing  the  strtrj«ct-egr«es~that 
inserting  a  variable  representing  production  rate  into  the  basic 
learning  curve  equation  produces  statistically  unreliable  results. 

J^UFSC  knows  about  and  accepts  the  statistical  limitation  because  the 
model  was  developed  to  provide  a  good  approximation  of  the  change 
in  cost  that  result  from  a  production  rate  change. 


INTRODUCTION 


The  call  usually  comes  late  Friday  afternoon,  but  it  is  always 
the  same.  The  Program  Element  Monitor  (PEM)  is  desperately 
seeking  the  impact  of  a  reduction  and/or  slip  in  the  buy  schedule 
of  a  major  weapon  system  program.  The  question,  from  some 
congressional  panel  or  Air  Force  exercise,  is  put  to  the  Systems 
Officer  (SYSTO)  at  HQ  Air  Force  System  Command  (AFSC).  It  is  up 
to  the  SYSTO  to  see  that  an  answer  is  found.  The  answer  is 
usually  delivered  within  two  days. 

HQ  AFSC,  through  its  Cost  Methods  Improvement  Program  (CMIP), 
contracted  with  The  Analytical  Sciences  Corporation  (TASC)  to 
develop  a  model  to  evaluate  the  impact  of  production  rate  changes 
on  program  cost.  The  model  is  to  provide  answers  the  same  day 
the  "what  if"  exercise  is  initiated.  Some  of  the  ground  rules 
used  in  developing  the  model  were  that  the  final  product  be  user 
friendly,  use  readily  available  information  as  inputs,  and  be 
free  of  contractor  support  after  delivery. 

BACKGROUND 

Techniques  for  estimating  recurring  costs,  which  include  the 
production  rate,  are  receiving  more  and  more  attention.  AFSC's 
model  uses  the  basic  equation  for  incorporating  production  rate 
(see  equation  1).  Reviewing  some  of  the  literature  shows  much 
criticism  aimed  at  the  effects  of  multicollinearity  induced  by 
the  introduction  of  the  production  rate  variable.  Intuitively, 
production  rate  and  cumulative  quantity  appear  to  be  highly 
related  and  influence  the  significance  each  variable  has  in 
predicting  unit  or  lot  costs.  For  this  reason  a  nonlinear 
approach  was  used  in  the  AFSC  model  for  estimating  model 
parameters. 

This  article  was  written  as  a  means  of  providing  an  understanding 
of  the  details  of  the  AFSC  model  in  order  that  it  may  be  better 
understood  as  an  estimating  tool. 

The  model  is  formatted  into  three  sections:  (1)  Data  Base 
Analysis,  (2)  Analysis  of  an  AFSC  Major  Program,  and  (3)  new 
Program  Analysis.  The  Historical  Program  Data  Base  is 
broken  into  categories  which  include:  fighters,  bombers,  tactical 
missiles,  tactical  armaments,  helicopters,  and  electronics/ 
avionics.  Within  each  category  is  a  listing  of  weapon  system 
programs.  For  example: 


FIGHTERS 


PROGRAM 

NAME 

F-IOO 
F-101 
F-102 
F-106 
F-15  A/B 
F-15  C/D 
F-15  E 
F-16  C/D 
A-10 


LEARNING 

CURVE 

0.930 

0.920 

0.740 

0.950 

0.900 

0.930 

0.940 

0.990 

0.990 


PROD  RATE 
PARAMETER 

0.970 

1.000 

1.000 

0.820 

1.000 

0.750 

0.960 

0.990 

0.890 


FIRST  UNIT 
COST  ($M) 

6.7 

7.1 

20.4 

14.3 

26.4 

24.6 

26.6 
17.9 
10.2 


HIGHEST 
PROD  RATE 


593 

309 

562 

165 

108 

97 

96 

180 

144 


In  deriving  the  learning  curve  and  production  rate  parameters  in 
the  Historical  Data  Base,  a  nonlinear  regression  technique  is 
used  to  reduce  midpoint  bias  and  multicollinearity.  Both  will  be 
discussed  in  more  detail  later.  The  value  that  minimized  the  sum 
of  the  squared  error  (SSE)  was  used.  The  values  used  may  not 
represent  the  most  logical  since  a  global  search  of  all  minimums 
was  not  conducted.  It  is  possible  that  other  parameters  may 
exist  which  are  more  representative  of  the  program’s  true 
parameters. 

The  Major  AFSC  Programs  section  contains  10  of  the  most  current 
AFSC  programs  allowing  for  easier  analysis  as  official  program 
funding  profiles  are  stored  here.  This  means  the  analyst  will 
not  have  to  retrieve  information  from  the  Historical  Data  Base 
prior  to  any  analysis.  The  New  Program  Analysis  section  allows 
analysis  of  systems  not  in  the  Major  AFSC  Programs  section,  as 
well  as  new  developing  programs.  Once  an  analysis  is  completed 
the  analyst  can  do  a  rebaseline  of  the  production  schedule  or  a 
sensitivity  analysis.  The  sensitivity  analysis  can  be  done  on 
the  first  unit  cost,  learning  curve  slope,  production  rate 
parameter,  or  optimal  production  rate.  The  production  schedule 
and  four  items  in  the  sensitivity  analysis  represent  the  major 
inputs  to  the  model.  In  addition,  inflation  indicies  are  input 
by  the  analyst  for  Then  Year  calculations. 


EXAMINING  THE  MODEL 


The  AFSC  model  is  based  on  an  extension  of  learning  curve  theory, 
using  a  variable  for  the  production  rate  parameter  as  shown  in 
equation  1. 

(Equation  1) 

Y  =  AX'blTc 


where : 


Y  =  cost  of  unit  X 
A  =  cost  of  the  first  unit 
X  =  cumulative  quantity  produced 
b  =  slope  of  the  learning  curve 
R  =  production  rate  in  effect  (yearly  qty) 
c  =  slope  of  the  production  rate  curve 
A  =  notation  for  an  exponential  function 
e.g.  x~2  is  X  squared 


The  influence  an  increasing  production  rate  has  on  decreasing  the 
unit  cost  is  limited  [1].  This  idea  is  identical  to  the  long  run 
average  cost  curve  in  economic  theory.  In  this  case  a  minimum 
cost  point  is  determined  by  the  manufacturer  prior  to  production. 
The  point  is  based  on  plant  capacity,  capital  investment  and 
manpower  requirements,  anticipated  quantities  and  production 
rates,  and  requirements  specified  by  the  government.  Therefore, 
the  formulation  considers  unit  cost  to  be  a  function  of 
production  rate  and  cost  improvement,  thus  yielding  a  three 
dimensional  curve  such  as  the  one  shown  in  figure  2  [1], 


Recurring  production  costs  are  estimated  by  taking  the  integral 
of  the  unit  cost  equation,  equation  1,  which  produces  the 
following  [1]: 

(Equation  2) 

Zi  =  ( A/B+1 )  )  [((N+Ki  +  0.5rB+1)  -  ( (N+0.5TB+1 )  R^C] 


This  equation  is  the  mathematical  basis  of  the  model  for 
estimating  all  recurring  production  costs.  However,  variable  A 
of  equation  2  is  not  the  first  unit  cost  —  it  is  the  Surface 
Initialization  Point  (SIP).  The  SIP  is  calculated  internally  by 
the  model  using  the  following  equation  [1]: 


(Equation  3) 

SIP  =  FIRST  UNIT  COST/ (TOOLED  TO  RATE)~c 


The  SIP  rather  than  the  first  unit  cost  is  used  to  initialize  the 
graph  shown  in  figure  2.  The  SIP  represents  a  theoretical  first 
unit  cost  if  the  unit  were  produced  at  the  tooled  to  rate.  In 
calculating  the  SIP  the  tooled  to  rate  is  used  because  the  first 
let  quantity  arbitrarily  weights  the  first  let  cost  calculation 
and  skews  the  rest  of  the  estimate. 
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Once  the  recurring  cost  is  computed,  Engineering  Change  Order 
(ECO),  nonrecurring,  peculiar  support,  and  the  program  unique 
costs  are  added  to  give  a  total  program  cost  similiar  to  that 
found  on  an  Air  Force  Form  1537.  Figure  1  is  a  sample  output 
showing  the  above  compilation. 

FIGURE  1 

COST  ELEMENT  FISCAL  YEAR  TOTAL 


7X 

7X 

8X 

8X 

8X 

8X 

QUANTITY 

XX 

XX 

XX 

XX 

XX 

XX 

XXX 

PME  (RECURRING) 

XX 

XX 

XX 

XX 

XX 

XX 

xxxx 

ECO 

X 

X 

XX 

XX 

X 

X 

XX 

NONRECURRING 

XX 

XX 

XX 

XX 

XX 

XX 

XX 

PECULIAR  SUPP  EQUIP 

XX 

XX 

XX 

XX 

XX 

XX 

XXX 

PROGRAM  UNIQUE 

X 

X 

X 

X 

X 

X 

XX 

TOTAL  BASE  YEAR 

XX 

XX 

XX 

XX 

XX 

XX 

xxxx 

INFLATION  INDEX 

X 

XX 

X 

X 

XX 

XX 

TOTAL  THEN  YEAR 

XX 

XX 

XX 

XX 

XX 

XX 

xxxx 

HISTORICAL  DATA  BASE 

In  deriving  the  Historical  Data  Base,  a  nonlinear  regression  was 
used  to  find  the  first  unit  cost,  learning  curve  and  production 
rate  slopes.  As  stated  earlier,  the  nonlinear  technique  was  used 
to  reduce  midpoint  bias  and  multicollinearity.  However,  the 
issue  of  dependence  between  production  rate  and  cum  quantity  is 
not  specifically  addressed.  Previous  studies  are  cited  as  a 
basis  for  the  nonlinear  approach.  First,  bias  is  introduced  with 
the  selection  of  the  lot  midpoints  —  averages  which  are  used  in 
the  log  linear.  Ordinary  Least  Squares  (OLS)  equation  used  to 
estimate  the  cost  improvement  parameters.  A  Rand  study  states 
the  OLS  approach  leads  to  biased  estimates  of  the  cost 
improvement  parameters  because  the  lot  midpoint  is  a  function  of 
the  improvement  curve  [7].  Further  support  is  found  in  a  study 
by  Goldberger,  indicating  that  logarithmic  transformation  biases 
the  estimation  of  the  first  unit  cost  [4].  Documentation  for 
the  model  does  not  show  the  extent  of  reduction  the  nonlinear 
approach  has  on  the  problem  of  multicollinearity.  As  such,  a 
true  evaluation  of  the  developer’s  claim  was  not  conducted, 
although  their  approach  has  an  intuitive  appeal. 
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If  true  independence  is  to  be  achieved  when  using  the  linear  OLS 
described  above,  then  the  covariance  of  the  variables  must  be 
equal  to  zero.  When  the  production  rate  equation,  equation  (1), 
is  transformed  and  the  least  squares  estimator  for  c  (the 
production  rate  slope)  is  found,  the  results  show  that  the 
covariance  is  not  equal  to  zero.  The  result  is  equal  to  the 
summation  of  X'  plus  the  summation  of 

R'jJV+JV  [2]s 

(Equation  1) 

Y  =  AX^bR'c 

transformed: 


InY  =  A  +  b(lnX)  +  c(lnR)  +  e 


(Equation  4) 


where : 


InY  =  logarithm  of  the  unit  cost  of  X 
InR  =  logarithm  of  the  production  rate 
lnX  =  logarithm  of  the  estimated  lot  midpoint  defined  as 
the  true  lot  midpoint  plus  some  error, 
e  =  random  error 
A,b,c  are  to  be  estimated 


Equation  (4)  can  now  be  rewritten  as  follows: 


(Equation  5) 

InY  =  A  +  b(2n(X  +/9))  +  C(lnR)  +  e 
Solving  for  the  least  squares  estimator  for  c  results  in: 

c  =  jTR.Sr-2  -JVx’Ex’r1 


JV~2JV~2  -  (JVyT2 

where : 


Y»  =  InY  -  InY 


X*  =  ln(X  +/3  )  -  ln(X  +/3) 
R*  =  InR  -  InR 


From  the  least  squares  estimator  of  c  comes  the  covariance 
^X'-V^R*  identified  earlier  which  is  not  equal  to  zero.  This 
indicates  that  true  independence  does  not  exist  for  the 
production  rate  equation  when  regressed  in  linear  form  and  why 
the  nonlinear  approach  was  used. 
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Further  compounding  the  problem  were  the  results  of  a  recent 
study  by  Sherbrooke  and  Associates  of  the  AIM-7F  (Sparrow 
missile)  showing,  through  linear  regression,  a  non-significant 
production  rate  parameter  [8].  Examining  the  results  of  the 
study  and  looking  at  the  difference  between  the  log  of  the 
production  rate  and  the  estimated  lot  midpoints  a  covariance  of 
0.84  is  found.  This  indicates  that  one  variable  may  assume  all 
explanatory  significance  of  the  other  when  the  two  variables  are 
highly  correlated.  In  other  words,  the  true  significance  of  the 
othe~  variable  can  not  be  evaluated  [2]. 

Because  of  this  additional  problem  a  nonlinear,  weighted  least 
squares  method  developed  by  Newton  was  used.  This  method  is 
equivalent  to  the  Gauss-Newton  method  contained  in  the 
Statistical  Analysis  System  (SAS)  computer  package  [1].  Using 
the  nonlinear  regression,  equation  (1)  was  solved  for  the 
learning  curve  slope  (b),  production  rate  slope  (e),  and  first 
unit  cost.  In  doing  so,  the  regression  produced  a  learning 
curve  slope  which  is  affected  by  the  production  rate  and  its 
slope . 

Evaluating  the  production  rate  slope  is  more  difficult  as  its 
value  is  calculated  and  has  little  historical  data  to  support  the 
values.  This  slope  indicates  to  what  degree  costs  would  change 
as  the  production  rate  changes.  This  is  certainly  germane  to 
recurring  costs;  however,  the  bottom  line  cost  will  be  determined 
by  the  allocation  of  the  overhead  costs.  Building  a  relationship 
between  production  rate  and  overhead  costs  for  estimating 
purposes  is  difficult  to  quantify.  The  relationship  would  be 
related  to  the  company's  overall  industrial  base  and  production 
rate  at  any  one  point  in  time.  Developing  this  relationship  was 
not  within  the  scope  of  effort  for  developing  the  AFSC  model. 
Follow-on  efforts  will  encompass  this  complex  relationship. 

To  update  the  Historical  Data  Base,  the  analyst  must  use  a 
nonlinear  program  outside  the  model  to  determine  the  values  for 
the  first  unit  cost,  learning  curve  and  production  rate  slopes. 

In  addition,  if  the  analyst  wants  to  use  any  of  the  historical 
learning  cur"e  or  production  rate  slopes  in  the  New  Program 
Analysis  segment,  the  model  only  provides  simple  averages.  More 
analytical  rigor  could  have  been  provided  by  using  a  regression 
of  the  historical  data  or  a  weighted  average  calculation.  This 
will  be  discussed  in  the  New  Program  Analysis  review. 


mm 


MAJOR  AFSC  PROGRAMS 


This  segment  of  the  model  contains  the  current  program  funding 
profile  for  major  AFSC  systems.  This  section  of  the  model 
includes  the  following  programs: 

1.  F-15C/D/E 

2.  B-1B 

3.  KC-10 

4.  F-16 

5.  HH-60D 

6.  JTIDS  (CLASS  II) 

7.  LANTIRN 

8.  HR  MAVERICK 

9.  HARM 

10.  T-46A 

The  data  for  these  systems  will  be  updated  periodically  to 
include  the  most  current  official  program  information. 

This  segment  of  the  model  allows  for  easier  analysis  as  the 
analyst  will  not  have  to  dig  through  the  Historical  Data  Base  to 
retrieve  learning  curve,  production  rate,  or  first  unit  cost 
information.  The  current  production  schedule  is  also  stored 
here,  and  program  cost  information  is  presented  in  base  year 
dollars.  The  analyst  need  only  input  changes  to  the  program 
information  in  order  to  conduct  any  analysis.  Once  the  changes 
are  input,  the  recurring  costs  are  calculated  using  equation  (2). 

NEW  PROGRAM  ANALYSIS 

In  this  segment  of  the  model  the  analyst  conducts  analysis  of 
new  programs  not  found  in  the  Major  AFSC  Program  segment.  This 
segment  of  the  model  requires  the  most  input  from  the  analyst. 

The  analyst  must  determine  what  learning  curve  and  production 
rate  slope  values  are  needed.  This  can  be  done  in  two  ways.  The 
first  is  from  previous  contractor  experience;  the  second  is  from 
the  Historical  Data  Base.  If  the  second  alternative  is  used,  the 
analyst  must  select  the  historical  systems  which  best  represent 
the  system  being  analyzed.  The  model  takes  the  information  for 
the  systems  selected  from  the  Historical  Data  Base  (reference 
Table  1)  and  computes  a  simple  average  for  the  learning  curve  and 
production  rate  slopes  respectively.  For  example,  a  new 
generation  ground  attack  aircraft  is  being  considered.  The 
analyst  determines  that  the  F-15E,  F-16E,  and  A-10  best  represent 
some  of  the  characteristics  of  the  new  system.  Based  on  the 
analyst’s  selection,  the  model  computes  an  average  from  the 
historical  learning  curve  and  production  rate  slopes.  The  new 
averages  are  the  values  used  as  inputs  for  the  New  Program 
Analysis. 


For  this  example,  the  simple  average  calculated  for  the 
production  rate  parameter  is  affected  by  a  low  value  for  the 
F-16E  production  rate  slope.  In  arriving  at  the  new  values,  the 
analyst  must  take  into  consideration  what  charaeteristic( s)  to 
focus  on  when  selecting  programs  from  the  Historical  Data  Base. 
This  is  important  as  it  may  help  explain  results  that  don't 
follow  "normal"  behavior.  In  this  instance  the  F-16E  production 
rate  slope  may  be  lower  than  expected  because  of  the  modernized 
plant  facility.  This  may  cause  results  to  be  different  than 
expected.  Regardless  of  what  the  analyst  focuses  on  as  the 
driving  characteristic,  a  simple  average  calculation  does  not 
appropriately  weight  a  system  such  as  the  F-16.  To  follow  the 
analytical  rigor  used  to  develop  the  historical  data  base,  a 
weighted  average  or  regression  calculation  could  be  used. 

After  determining  the  learning  curve  and  production  rate  slopes, 
the  analyst  must  input  the  remaining  data  elements.  These  are 
determined  outside  of  the  model  and  include  first  unit  cost, 
starting  year  of  production,  yearly  production  schedule,  program 
base  year,  and  inflation  indicies  (for  Then  Year  calculations). 
Recurring  costs  are  again  calculated  using  equation  (2).  Once 
the  output  is  presented,  the  analyst  can  then  perform  a 
rebaseline  or  sensivity  analysis. 

OUTPUT  REPORTS 


Output  reports  are  provided  for  the  Major  AFSC  Program  and  New 
Program  Analysis  segments  of  the  model.  Once  a  system  is 
selected  from  the  Major  AFSC  Program  data  base,  a  multiple  page 
(video  display  screen)  report  is  provided  as  shown  by  the  example 
in  figure  3.  The  information  provided  in  this  report  is  the  most 
current  program  funding  profile. 


FIGURE  3 


McDonnell  Douglas 
St.  Louis  MO 


COST  IMPROVEMENT  RATE:  0.903 
PRODUCTION  RATE  PARAMETER:  0.906 
TOOLED  RATE:  144 

INITIAL  PRODUCTION  YEAR:  1984 
PROGRAM  BASE  YEAR:  1983 


MAJOR  PROGRAM  DATA  BASE 


PAGE  1  OF  3 


System:  F-15C/D/E 
Type:  Fighter 


Prime  contractor: 
Primary  Facility: 


FIRST  UNIT  COST 


55.024  MILLION 
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FIGURE  3  (cont.) 


PRODUCTION  SCHEDULE  (COSTS  IN  MILLIONS) 


FY84 

FY85 

FY86 

FY87 

FY88 

QUANTITY 

36 

48 

60 

72 

96 

PME 

906.4 

1151.1 

1380.8 

1362.1 

1354.8 

ECO 

15.4 

19.3 

23.0 

22.8 

22.6 

NON-RECURRING 

104.3 

52.6 

22.6 

8.9 

4.3 

PEC  SUPP  EQP 

395.5 

384.7 

390.4 

343.5 

297.8 

PROG  UNIQUE 

144.9 

154.5 

130.4 

124.1 

117.2 

TOTAL  BASE  YEAR 
INFLATION  INDEX 
TOTAL  THEN  YEAR 

If  a  rebaseline  of  the  program  is  done  a  similiar  report  is 
provided.  The  difference  being  that  the  program  and  contractor 
specific  information  is  not  included,  as  can  be  seen  in  figure  4 
below.  The  Base  Year  and  Then  Year  deltas  show  the  change  from 
the  original  to  the  rebaseline.  The  last  page  of  the  rebaseline 
report  is  a  sample  POM  submission  based  on  the  new  schedule  (see 
figure  5).  Included  in  all  of  the  reports  are  TY  costs,  based  on 
the  indicies  provided  by  the  analyst. 

FIGURE  4 
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SYTEM:  KC-10 

Tooled 

Rate:  10 

TYPE;  Transport 

Program 

Base  Year 

Fiscal  Year 

1978 

1979 

1980 

1981 

1982 

Lot  size 

2 

4 

6 

4 

8 

Inflation  indicies 

1.335 

1.455 

1.586 

1.727 

1.820 

Recurring 

97.1 

151.4 

220.7 

146.1 

286.5 

ECO 

7.0 

6.1 

5.3 

6.1 

4.4 

Nonrecurring 

0.0 

47.7 

0.0 

0.0 

0.0 

Peculiar  Support 

0.0 

3.6 

6.8 

11.5 

151.7 

Program  Unique 

-19.2 

-39.2 

-60.0 

-40.0 

-56.0 

Base-year  Total 

66.8 

169.6 

172.8 

123.8 

386.6 

Then-year  Total 

89.2 

246.7 

274.0 

213.8 

703.6 

Base-year  Delta 

N/A 

51.4 

47.7 

-53.7 

122.6 

Then-year  Delta 

N/A 

74.8 

75.7 

-92.8 

223.2 
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SYSTEM:  KC-10  Tooled  Rate:  10 

TYPE:  Transport  Program  base  year:  1976 

POM  SUBMISSIONS 


FY85 

and  prior 

FY86 

FY87 

FY88 

FY89 

FY90 

To 

Complete 

BASE  YEAR 

919.6 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

THEN  YEAR 

1,527.4 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

When  doing  a  sensitivity  analysis  the  model  will  prompt  for  an 
upper  and  lower  bound,  and  increment  of  increase.  The  output 
provides  costs  for  each  increment  between  the  bounds  at  a  summary 
and  per  increment  level.  The  increment  level  detail  is  given  by 
year  showing  the  change  from  the  comparison  value  as  an  absolute 
and  percent  amount  (see  figures  6  and  7). 

FIGURE  6 
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SYSTEM: 

KC-10 

Sensitivity  parameter: 

FIRST  UNIT 

COST 

TYPE: 

Transport 

Comparison  value: 

38.87 

SCHEDULE: 

Rebaselined 

Value  range  from  35  to  55  by  10 

PARAMETER 

TOTAL 

PME  ($M)  DELTA  PME  ($M) 

PCT  CHANGE 

VALUE 

Base 

Then  Base  Then 

Base 

Then 

35. 

793.7 

1,301.9  -90.1  -147.8 

-10.2 

-10.2 

45. 

1,020.5 

1,673.8  136.7  224.2 

15.5 

15.5 

55. 

1,247.2 

2,045.8  363.4  596.1 

41.1 

41.1 

FIGURE  7 

PARAMETER  SENSITIVITY  REPORT 
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;  2  OF 

SYSTEM: 

KC-10 

Sensitivity  parameter: 

FIRST  UNIT 

COST 

TYPE: 

Transport 

Comparison  value:  38.87 

SCHEDULE: 

Rebaselined 

Value  range  from  35  to  55  by  10 

PARAMETER 

VALUE:  35. 

(Cost  units  are  millions 

of  dollars 

.) 

FY78 

FY79 

FY80 

FY81 

FY82 

BASE 

YEAR: 

Costs: 

71.0 

135.9 

198.2 

131.2 

257.3 

Changes: 

-8.1 

-15.4 

-22.5 

-14.9 

-29.2 

Percent: 

-10.19 

-10.19 

-10.19 

-10.19 

-10.19 

THEN 

YEAR: 

Costs: 

94.8 

197.8 

314.4 

226.7 

468.3 

Changes: 

-10.8 

-22.5 

-35.7 

-25.7 

-53.2 

Percent: 

-10.19 

-10.19 

-10.19 

-10.19 

-10.19 
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VALIDATION  OF  THE  MODEL 


Before  the  model  is  distributed  to  the  field  for  implementation, 
validation  of  the  model’s  output  will  be  conducted  by  HQ  AFSC. 

The  process  for  validation  has  started  and  uses  a  concept  similiar 
to  one  identified  by  Graver  [5].  This  technique  "scores"  the  results 
of  the  model. 

The  technique  for  scoring  will  be  done  using  the  model’s  current 
data  base  and  the  official  Air  Force  Fiscal  Year  87  Program 
Objective  Memorandum  (POM).  Using  the  current  data  base,  cost 
estimates  for  the  quantities  in  the  87  POM  will  be  made 
and  compared  to  the  dollar  amounts  in  the  POM.  Then  the  estimate 
will  be  divided  by  the  POM  amount  to  develop  a  score.  Obviously 
the  closer  the  ratio  is  to  one  the  better  the  score.  The  results 
of  the  scoring  exercise  are  provided  in  figure  8.  An  acceptance 
band  between  .80  and  1.20  was  chosen  as  a  good  score.  If  the  model 
scores  are  consistently  within  the  acceptance  band  the  output  of 
the  model  will  not  require  extensive  calibration.  If  the  results 
are  eratic  an  explanation  is  required.  For  example,  the  HR 
Maverick  graph  in  figure  8  shows  a  dip  in  its  score  from  FY  85, 

.89,  to  F¥  86,  .67.  The  reason  for  the  dip  is  a  change  in  quantity 
from  the  model’s  data  base  to  the  87  POM.  Although  the  quantity 
was  reduced  by  6,000  missiles,  the  dollars  associated  with  the 
quantity  were  not  reduced  by  a  proportionate  amount.  If  you  adjust 
the  model  results  by  the  increase  in  the  unit  cost  the  score  becomes 
.84  not  .67. 

If  the  extreme  variances  in  the  model’s  output  can  be  explained 
then  AFSC  can  use  and  understand  the  model  on  a  routine  basis.  The 
above  scoring  example,  as  all  scores,  was  based  on  comparing 
recurring  costs  only.  This  was  done  because  of  the  wide  range 
between  the  model’s  data  base  and  the  87  POM  for  the  other  costs. 

One  "what  if"  exercise  was  conducted  earlier  this  year  with 
Aeronautical  Systems  Division’s  (ASD)  director  of  Cost  Analysis 
for  quantity  changes  to  the  F-15  program.  The  exercise  done  by 
the  Program  Office  showed  a  decrease  in  unit  cost  from  5  to  10 
percent.  When  the  same  changes  were  calculated  using  the  Rate 
Model,  a  change  in  unit  cost  from  4  to  8  percent  was  experienced. 

The  results  were  encouraging  but  further  validation  is  needed. 


FIGURE  8 
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SUMMARY 


Seme  of  the  theoretical  ideas  behind  the  model  may  still  be 
questioned;  however,  these  problems  plague  the  basic  equation  used 
to  incorporate  production  rate.  The  model  does  appear,  based 
on  the  validation  process,  to  provide  a  good  estimate  of 
recurring  cost  for  "what  if"  exercises.  In  addition  the  model 
provides  a  good  foundation  for  future  expansion. 

AFSC  has  already  started  a  study  to  incorporate  effects  of  changes 
in  the  production  rate  on  overhead.  This  complex  relationship  is  a 
key  to  determining  the  final  program  cost.  Some  studies  such  as 
Gulledge’s  [6]  are  a  good  attempt  to  capture  factors  other  than 
cumulative  quantity  and  production  rate  effecting  cost.  Additionally, 
AFSC  has  sponsored  a  thesis  undertaken  by  Bolton  [3]  at  the  Air  Force 
Institute  of  Technology.  This  effort  is  to  examine  the  reduction  in 
multieollinearity  as  a  result  of  estimating  parameters  with  the 
nonlinear  approach.  In  addition,  new  equations  will  be  examined  to 
see  if  the  dependency  between  the  variable  can  be  eliminated  and  true 
independence  found.  This  effort  should  be  completed  by  September 
1985. 

Areas  for  future  consideration  include  the  addition  of  a 
nonlinear  program  to  update  the  data  base.  A  more  rigorous 
approach  for  developing  learning  curve  and  production  rate  slopes 
used  in  the  New  Program  Analysis  may  be  developed  to  replace  the 
simple  average  calculation.  An  exploration  of  all  minimum  values 
resulting  from  the  nonlinear  regression  should  be  undertaken  to 
find  the  most  logical  value.  Lastly,  the  treatment  of  contractor 
overhead  must  be  developed  from  the  parametric  standpoint.  The 
efforts  are  some  of  the  areas  of  future  expansion  allowed  by  the 
model . 
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